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WHAT IS CLAIMED IS: 

1. A method of assessing the reliability of a metric, comprising the 
steps of: 

(a) evaluating the metric with respect to each of at least one 
key factor to produce a factor score for each key factor; and 

(b) when there is more than one factor score, combining the 
factor scores to form a final weighted numerical assessment for the metric. 

2. The method of claim 1, wherein said step (a) comprises the step 
of evaluating the metric with respect to a source of data that is used in developing 
the metric. 

3. The method of claim 2, wherein said step of evaluating the metric 
with respect to the source of data comprises the steps of: 

i) scoring the number of run failures over a previous 
predetermined interval of time; 

ii) scoring the number of system failures over the interval; 

iii) scoring the number of partial runs over the interval; and 

iv) summing the scores for the run failures, system failures, 
and partial runs, to form a data source factor score. 

4. The method of claim 1, wherein said step (a) comprises the step 
of evaluating the metric with respect to business rules used in developing the 
metric. 

5 . The method of claim 4, wherein said step of evaluating the metric 
with respect to business rules comprises the steps of: 

i) scoring the presence of clear business rules; 
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ii) scoring the extent of definition and documentation of 

business rules; 

iii) scoring the extent to which a method used to develop the 
metric reflects the business rules; 

iv) scoring the consistency of the business rules with corporate 
strategies and processes; and 

v) summing the scores from said steps i) through iv) to form 
a business rules factor score. 

6. The method of claim 1, wherein said step (a) comprises the step 
of evaluating the process used in the production of the metric. 

7. The method of claim 6, wherein said step of evaluating the process 
used in the production of the metric comprises the steps of: 

i) scoring the presence of manual processing in retrieving 
data used to produce the metric; 

ii) scoring the presence of manual processing in populating 
a data structure; 

iii) scoring the presence of manual processing in determining 

the metric; and 

iv) summing the scores from said steps i) through iii) to form 
a manual processing factor score. 

8. The method of claim 1, wherein said step (a) comprises the step 
of evaluating the metric with respect to mathematical stability of the metric. 

9. The method of claim 8, wherein said step of evaluating the metric 
with respect to mathematical stability comprises the steps of: 

i) scoring the use of a mathematical calculation to derive the 

metric; and 
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ii) scoring the use of a reliable non-mathematical method to 
derive the metric; and 

iii) summing the scores from said steps i) and ii) to form a 
mathematical stability factor score. 

10. The method of claim 1, wherein said step (a) comprises the step 
of evaluating the metric with respect to integrity of a data source. 

1 1 . The method of claim 1 0 , wherein said step of evaluating the metric 
with respect to integrity of the data source comprises the steps of: 

i) determining whether the data source has been validated; 

ii) if the data source has been validated, scoring the recency 
and automation of the validation to form an integrity factor score; and 

iii) if the data source has not been validated, assigning the 
value of zero as the integrity factor score. 

12, The method of claim 1, wherein said step (a) comprises the step 
of evaluating the metric with respect to supporting detail for the metric. 

13, The method of claim 12, wherein said step of evaluating the metric 
with respect to supporting detail comprises the steps of: 

i) determining a length of time for which valid supporting 
historical detail has been available; 

ii) determining a degree of interruption of the valid supporting 
historical detail; and 

iii) scoring, in aggregate, the length of time for which valid 
supporting historical detail is available and the degree of interruption of the valid 
supporting historical detail, to form a supporting detail factor score. 
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14. The method of claim 1 , wherein said step (b) comprises the step 
of adding together the at least one factor score to form the final weighted 
numerical assessment of the metric. 

15. A computer program product comprising a computer usable 
medium having computer readable program code means embodied in said 
medium for causing an application program to execute on a computer to assess 
the reliability of a metric, said computer readable program code means 
comprising: 

(a) first computer readable program code means for causing 
the computer to evaluate the metric with respect to each of at least one key factor 
to produce a factor score for each key factor; and 

(b) second computer readable program code means for causing 
the computer to, when there is more than one factor score, combine the factor 
scores to form a final weighted numerical assessment for the metric. 

16. The computer program product of claim 15, wherein said first 
computer readable program code means comprises computer readable program 
code means for evaluating the metric with respect to a source of data that is used 
in developing the metric. 

17. The computer program product of claim 16, wherein said 
computer readable program code means for evaluating the metric with respect to 
the source of data comprises: 

i) computer readable program code means for scoring the 
number of run failures over a previous predetermined interval of time; 

ii) computer readable program code means for scoring the 
number of system failures over the interval; 

iii) computer readable program code means for scoring the 

number of partial runs over the interval; and 
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i v) computer readable program code means for summing the 
scores for the run failures, system failures, and partial runs, to form a data source 
factor score. 

18. The computer program product of claim 15, wherein said first 
computer readable program code means comprises computer readable program 
code means for evaluating the metric with respect to business rules used in 
developing the metric. 

19. The computer program product of claim 18, wherein said 
computer readable program code means for evaluating the metric with respect to 

business rules comprises: 

i) computer readable program code means for scoring the 

presence of clear business rules; 

ii) computer readable program code means for scoring the 

extent of definition and documentation of business rules; 

iii) computer readable program code means for scoring the 
extent to which a method used to develop the metric reflects the business rules; 

iv) computer readable program code means for scoring the 
consistency of the business rules with corporate strategies and processes; and 

v) computer readable program code means for summing the 
scores from said computer readable program code means i) through iv), to form 
a business rules factor score. 

20. The computer program product of claim 15, wherein said first 
computer readable program code means comprises computer readable program 
code means for evaluating the metric with respect to a process used in producing 
the metric. 
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21. The computer program product of claim 20, wherein said 
computer readable program code means for evaluating the metric with respect to 
the process used in producing the metric comprises: 

i) computer readable program code means for scoring the 
presence of manual processing in retrieving data used to produce the metric; 

ii) computer readable program code means for scoring the 
presence of manual processing in populating a data structure; 

iii) computer readable program code means for scoring the 
presence of manual processing in determining the metric; and 

iv) computer readable program code means for summing the 
scores from said steps i) through iii) to form a metric production process factor 
score. 

22. The computer program product of claim 15, wherein said first 
computer readable program code means comprises computer readable program 
code means for evaluating the metric with respect to mathematical stability of the 
metric. 

23. The computer program product of claim 22, wherein said 
computer readable program code means for evaluating the metric with respect to 
mathematical stability comprises: 

i) computer readable program code means for scoring the use 

of a mathematical calculation to derive the metric; and 

ii) computer readable program code means for scoring the use 
of a reliable non-mathematical method to derive the metric; and 

iii) computer readable program code means for summing the 
scores from said steps i) and ii) to form a mathematical stability factor score. 
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24. The computer program product of claim 15, wherein said first 
computer readable program code means comprises computer readable program 
code means for evaluating the metric with respect to integrity of the data source. 

25. The computer program product of claim 24, wherein said 
computer readable program code means for evaluating the metric with respect to 
integrity of the data source comprises: 



recency and automation of the validation to form an integrity factor score if the 
data source has been validated; and 



value of zero as the integrity factor score if the data source has not been validated. 

26. The computer program product of claim 15, wherein said first 
computer readable program code means comprises computer readable program 
code means for evaluating the metric with respect to supporting detail for the 
metric. 

27. The computer program product of claim 26, wherein said 
computer readable program code means for evaluating the metric with respect to 
supporting detail comprises computer readable program code means for scoring, 
in aggregate, the length of time for which valid supporting historical detail has 
been available and the degree of interruption of the valid supporting historical 
detail, to form a supporting detail factor score. 

28. The computer program product of claim 15, wherein said second 
computer readable program code means comprises computer readable program 
code means for adding together the at least one factor score to form the final 
weighted numerical assessment of the metric. 
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29. A system for assessing the reliability of a metric, comprising: 

(a) means for receiving an evaluation of the metric with 

respect to each of at least one key factor; 

(b) means for producing a factor score for each key factor; 
and 

(c) where there is more than one factor score, means for 
combining the factor scores to form a final weighted numerical assessment for the 
metric. 
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